# Image Restoration

HDR is a new technology focused on restoring damaged historical documents, aimed at predicting their original appearance. This technology utilizes the HD28K large-scale dataset and the diffusion-based network, DiffHDR, to address various damages including character loss, paper deterioration, and ink erosion. The primary advantage of HDR is its ability to accurately capture character content and style while ensuring coherence with the background of the restoration area. This technology not only restores damaged documents but also extends to document editing and text block generation, showcasing high flexibility and generalization capabilities. HDR holds significant importance for preserving invaluable culture and civilization.

BrushEdit is an advanced, unified AI agent for image restoration and editing. It combines multimodal large language models (MLLMs) and image restoration models to achieve automated, user-friendly, and interactive free-form instruction editing. The system employs MLLMs and a dual-branch image restoration framework to perform editing category classification, primary object recognition, mask generation, and area restoration. Extensive experiments show that this framework effectively integrates MLLMs and restoration models, achieving superior performance across seven key metrics, including mask area retention and editing effect consistency.

InstantIR is a blind image restoration method based on diffusion models that can handle unknown degradation problems during testing, enhancing the model's generalization capabilities. This technology dynamically adjusts generation conditions by generating reference images during inference, thereby providing robust generation conditions. Key advantages of InstantIR include the ability to restore details in extremely degraded images, delivering realistic textures, and enabling creative image restoration through text descriptions. This technology has been jointly developed by researchers from Peking University, the InstantX team, and The Chinese University of Hong Kong, with sponsorship support from HuggingFace and fal.ai.

Ideogram Canvas is an infinite creative canvas designed for organizing, generating, editing, and combining images. It supports uploading your own images or generating new ones within the canvas, utilizing industry-leading Magic Fill (image restoration) and Extend (image expansion) tools for seamless editing, expansion, or combination. Particularly suited for graphic design, it offers advanced text rendering and precise prompt adherence, enabling a flexible and iterative process to bring your creative visions to life. Ideogram Canvas boasts an AI-driven innovative interface intended to enhance your iterative creative process. Whether you're an experienced designer or just starting your design journey, our flexible platform allows you to transform your visions into reality with astonishing speed and accuracy, pushing the boundaries of creative freedom.

FLUX.1-dev-Controlnet-Inpainting-Beta is an image restoration model developed by Alibaba's creative team, featuring significant improvements in image restoration. It supports direct processing and generation at 1024x1024 resolution without the need for additional upscaling, providing higher quality and more detailed outputs. The model has been fine-tuned to capture and reproduce more details in the restoration areas, with enhanced prompt explanations that allow for more precise control over the generated content.

PMRF (Posterior-Mean Rectified Flow) is a newly proposed image restoration algorithm designed to address the distortion-perception quality trade-off in image restoration tasks. By combining posterior mean and rectified flow approaches, it presents an innovative image restoration framework capable of reducing image distortion while ensuring perceptual quality.

Posterior-Mean Rectified Flow (PMRF) is an innovative image restoration algorithm that minimizes mean square error (MSE) while ensuring the realism of images by optimizing posterior mean predictions and the rectified flow model. The PMRF algorithm is both simple and efficient, theoretically grounded in optimizing posterior mean predictions (minimum mean square error estimates) to match the true image distribution. It excels in image restoration tasks, effectively tackling various degradation issues such as noise and blurring, while providing good perceptual quality.

FLUX.1-dev-Controlnet-Inpainting-Alpha is an AI image restoration model released by the AlimamaCreative Team, specifically developed to repair and fill in missing or damaged areas of images. This model performs optimally at a resolution of 768x768, delivering high-quality image restoration. As an alpha version, it showcases advanced technology in the field of image restoration and is expected to provide even better performance with further training and optimization.

FLUX-Controlnet-Inpainting is an image restoration tool released by the Alibaba Mama Creative Team, based on the FLUX.1-dev model. This tool utilizes deep learning technology to restore images, filling in missing sections and suitable for image editing and enhancement. It performs best at a resolution of 768x768, providing high-quality image restoration results. Currently, the tool is in the alpha testing phase, with future updates planned.

Diffree is a text-guided image restoration model capable of adding new objects to images based on text descriptions while maintaining background consistency, spatial appropriateness, and the quality and relevance of the objects. This model was trained on the OABench dataset, utilizing a stable diffusion model along with an additional mask prediction module, enabling it to uniquely predict the locations of new objects for text-guided object addition.

AI Image Extender is a tool that leverages AI technology to extend image boundaries by generating new content that seamlessly blends with existing images, thereby enhancing their visual extension. This product employs advanced AI algorithms to intelligently recognize image content and create naturally transitioning expanded areas, making it suitable for various scenarios that require image extension or background generation.

SwinIR is the official PyTorch implementation of an image restoration model based on Swin Transformer. It achieves state-of-the-art performance on tasks such as classic, lightweight, and real-world image super-resolution, grayscale/color image denoising, and JPEG compression artifact removal. It consists of shallow feature extraction, deep feature extraction, and high-quality image reconstruction, with excellent performance and parameter optimization.

InstructIR accepts images and human-written instructions as input, performing integrated image restoration through a single neural model. It has achieved state-of-the-art results across multiple restoration tasks, including image denoising, rain removal, defogging, deblurring, and low-light image enhancement. ?? Get started with the demonstration tutorial. Visit our GitHub for more information. Disclaimer: Please note that this is not a product, and you may notice certain limitations. This demonstration requires input of images with certain degradations (blur, noise, rain, low light, fog) and a prompt indicating the operation to be performed. The application may crash if input high-resolution images (2K, 4K) are used due to GPU memory limitations. The model is primarily trained on synthetic data, which may result in suboptimal performance on real-world complex images. However, it performs surprisingly well on real-world foggy and low-light images. You can also try general image enhancement prompts (e.g., 'polish this image', 'enhance color') to see how it improves color clarity.

This paper proposes a simple and effective personalized image restoration method, named Dual-Pivot Tuning. The method consists of two steps: 1) Fine-tuning a conditional generative model to leverage the conditional information in the encoder for personalization; 2) Fixing the generative model and adjusting the parameters of the encoder to adapt to the strengthened personalized prior. This can generate natural images that preserve both personalized facial features and image degradation attributes. Experiments demonstrate that compared to non-personalized methods, this method can generate higher-fidelity facial images.

Inst-Inpaint is an image restoration algorithm that can estimate and remove objects to be deleted based on natural language input. This product offers the GQA-Inpaint dataset and a novel restoration framework called Inst-Inpaint, capable of removing objects from images based on textual prompts. The product provides various GAN and diffusion baselines and conducts experiments on both synthetic and real-world image datasets. It utilizes different evaluation metrics to measure the quality and accuracy of the models, demonstrating significant quantitative and qualitative improvements.